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Fariba  Fahroo 


BRIEF  DESCRIPTION  OF  PORTFOLIO: 

Development  of  optimization  and  discrete  mathematics 
for  solving  large,  complex  problems  in  support  of  future 
Air  Force  science  and  engineering,  command  and  control 
systems,  logistics,  and  battlefield  management 

LIST  SUB-AREAS  IN  PORTFOLIO: 

*  Analysis  based  optimization 

*  Continuous  and  Discrete  Search  methods 

*  Dynamic,  Stochastic  and  Simulation  Optimization 

*  Combinatorial  Optimization 
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Program  Overview 


Optimization  is  at  the  heart  of  so  many  scientific  fields  and  applications  such 
as  engineering,  finance,  mathematical  biology,  resource  allocation,  network 
theory,  numerical  analysis,  control  theory,  decision  theory,  and  so  on. 

A  General  Optimization  Problem  Formulation 


Maximize  (or  Minimize)  f(x) 


Subject  to 


g(x)  <  0 
h(x)  =  0 


where  x  is  an  n-dimensionai  vector  and  g,  h  are  vector  functions.  Data  can 
be  deterministic  or  stochastic  and  time-dependent. 

Applications:*!)  Engineering  design,  including  peta-scale  methods,  2)Risk 
Management  and  Defend-Attack-Defend  models,  3)  Real-time  logistics,  4)  Optimal 
learning/machine  learning,  5)  Optimal  photonic  material  design  (meta-materials) , 
6)Satellite  and  target  tracking,  7)Embedded  optimization  for  control,  8) 


Network  data  mining 
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Program  Trends 


Analysis  based  optimization  -  Emphasis  on  theoretical  results 
to  exploit  problem  structure  and  establish  convergence 
*  properties  supporting  the  development  of  algorithms 

Search  based  optimization  -  New  methods  to  address  very  large 
►  continuous  or  discrete  problems,  but  with  an  emphasis  on  the 
mathematical  underpinnings,  provable  bounds,  etc 


Dynamic,  stochastic  and  simulation  optimization-  New 

algorithms  that  address  data  dynamics  and  uncertainty  and 
include  optimization  of  simulation  parameters 


Combinatorial  optimization  -  New  algorithms  that  address 
fundamental  problems  in  networks  and  graphs  such  as 
identifying  substructures 


Challenges:  Curse  of  Dimensionality,  Nonlinearity, 

Nondifferentiability,  Uncertainty  Acra^'  i 
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Dynamic  and  Stochastic  Methods 


*  Stochastic  optimization  addresses  the  problem  of 
making  decisions  over  time  as  new  information  becomes 
available. 

*  Applications:  operations  research,  economics,  artificial 
intelligence  and  engineering. 

*  Fields:  Approximate  dynamic  programming, 
reinforcement  learning,  neuro-dynamic  programming, 
optimal  control  and  stochastic  programming 

*  Dynamic  Optimization 

-  Powell,  Sen,  Magnanti  and  Levi 

-  Fleet  scheduling,  Refueling,  ADP,  Learning  methods,  Logistics 

*  Stochastic  Optimization  - 

-  Christos  Cassandras 
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Optimal  learning  and  approximate  dynamic 

programming 

^^^^Warrer^B^PoweM,  Princeton  Universjty^^^ 


Goal:  To  create  a  single,  general  purpose  tool  that  will  solve  sequential  decision 
problems  for  the  broadest  class  of  problems 

Challenges:  Complex  resources,  High-dimensional  resource  states,  Complex 
state  of  the  world  variables,  Belief  States 


*  Optimal  learning  for  general  decision  problems 

—  Learning  for  linear  programs  -  Optimal 
sequential  learning  for  graphs  and  linear 
programs,  dramatically  expands  the  classical 
lookup  table  model. 

—  Information  blending  -  Generalizes  learning 
problems  to  applications  where  information  is 
blended;  includes  extension  to  robust 
objectives. 

*  Approximate  dynamic  programming 

—  Fast,  semiparametric  approximation  method 
avoids  search  for  basis  functions.  Useful  for 
stochastic  search  and  ADP. 

*  Contributions  at  the  learning/ADP  interface 

—  Learning  with  a  physical  state  -  bridges  ADP 
and  learning  literature. 

—  Knowledge  gradient  for  semi-parametric  beliefs 
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Recent  Transitions  -  2012  BRI 

Warren  B.  Powell  (Princeton  University) 


Optimal  learning  for  nano-bio 
technologies  -  Funded  by  AFOSR 
Natural  Materials,  Systems  and 
Extremophiles  Program  (Hugh  de 
Long) 


SiNx  -300nm  (250°C) 

Or  -SOnrn  (room  T) 
n*  a-SiiH  -20nm  (230°C) 
a-Si:H  -150nm  (23(FC  to  ■ 
SiNx  ~3Q0nm  (350°c) 

Cr  ^80nm  (room  T) 


5$ 


—  The  goal  of  this  research  is  to  use 

optimal  learning  for  the  sequential  | 

design  of  experiments  in  the  physical  j 

CO 

sciences,  with  a  special  focus  on  nano-  I 
bio. 

—  Initial  project  addressing  design  of 
nanocrystalline  silicon  for  high- 
performance/low-power  transistor  circuit 
technology  on  flexible  substrates 

—  Developing  interactive  tools  for  belief 
extraction  which  is  used  to  compute 
knowledge  gradient. 
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OPTIMAL  PERSISTENT  SURVEILLANCE  THROUGH 

COOPERATIVE  TEAMS 
C.G.  Cassandras,  Boston  University 


OPTIMAL  PERSISTENT  SURVEILLANCE  PROBLEM: 

Unlike  the  COVERAGE  PROBLEM,  the  environment  is  too  large  to 
be  fully  covered  by  a  stationary  team  of  nodes  (vehicles,  sensors, 
agents).  Instead: 

-  all  areas  of  a  mission  space  must  be  visited  and  sensed 
infinitely  often 

-  some  measure  of  overall  uncertainty  must  be  minimized 
What  is  the  optimal  cooperative  way  for  nodes  to  move? 


Dark  brown:  HIGH  uncertainty  White:  NO  uncertainty 


^  KEY  RESULT:  In  a  one-dimensional  mission  space: 

-  optimal  trajectories  are  oscillations  between  left  and  right 
direction-switching  points 

-  nodes  may  dwell  at  these  points  for  some  time  before 
reversing  their  motion 

-  these  left  and  right  points  and  associated  dwell  times  can  be 
efficiently  determined 

Proofs  based  on  combining  Optimal  Control  and 

Infinitesimal  Perturbation  Analysis  for  Hybrid  Systems 


Optimal  oscillatory  trajectories  of  2  nodes  over  a  mission 
space  [0, 20] 
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OPTIMAL  PERSISTENT  SURVEILLANCE  THROUGH 

COOPERATIVE  TEAMS 
C.G.  Cassandras,  Boston  University 


4  KEY  RESULT:  In  a  two-dimensional  mission  space: 

-  single-node  optimal  trajectories  are  elliptical,  not  straight  lines 

-  all  ellipse  parameters  can  be  efficiently  determined 


For  two  nodes,  is  a  single  ellipse  with  both  nodes  better 
or  two  ellipses  with  one  node  each? 


4  PRELIMINARY  RESULT:  A  single  ellipse 
with  two  nodes  outperforms  two  ellipses 
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Analysis-Based  Optimization 
Theory  and  Methods 


-  Mifflin  and  Sagastizabal 

•  New  theory  leading  to  rapid  convergence  in  non- 
differentiable  optimization 

-  Freund  and  Peraire 

•  SDP  for  meta-material  design 
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Goal:  Design  of  photonic  crystals  with 
multiple  and  combined  band  gaps 

Approach:  Solution  Strategies 

•  Pixel-based  topology  optimization 

•  Large-scale  non-convex  eigen  value 
optimization  problem 

•  Relaxation  and  linearization 
Methods:  Numerical  optimization  tools 

•  Semi-definite  programming 


•  Subspace  approximation  method 


•  Discrete  system  via  Finite  Element 
discretezation  and  Adaptive  mesh 
refinement 

Results:  Square  and  triangular  lattices 

•  Very  large  absolute  band  gaps 

•  Multiple  complete  band  gaps 


•  Much  larger  than  existing  results. 
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Fabrication-Adaptive  Optimization, 
Application  to  Meta-material  Design 

Freund  and  Peraire  (MIT) 

Original  Simple  FA  (fabrication-adaptive) 

Optimization  Optimization 


•  FA  optimization  arises  from  the  need  to 
post-modify  optimal  solution  x*,  without 
severely  compromising  objective  value. 


•  FA  often  leads  to  non -convex  problems. 


•  Design  strategy  for  photonic  crystals: 
Nonlinear  Program  ->  Semidefinite 
Program  ->  Linear  Fractional  Program. 


^A^A 

^A^A 


L 


*  Submitted  to  Operations  Research 


5? 
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(a)  :  original  optimal  design; 

Gap(Xo)  =  43.9% 


(b)  yQ :  manual  (5%)  modification 
of  x*  ;  Gap(y0 )  =  28.8% 


(c)  XpA:  FA  optimal  design  with 
5=  5%;  Gap(xfA)  =  34.2% 


(d)  yFA:  manual  (0.8%)  modification 
of  xpA;  Gap(yFA)  =  32.9% 


DISTRIBUTION  STATEMENT  A  -  Unclassified,  Unlimited  Distribution 


12 


Combinatorial  Optimization 


Butenko  (Texas  A&M),  Vavasis  (U.  Waterloo), 
Krokhmal  (U.  Iowa),  Prokopyev  (U.  Pittsburgh) 

Underlying  structures  are  often  graphs;  analysis  is 
increasingly  continuous  non-linear  optimization 

Interested  in  detecting  cohesive  (tightly  knit)  groups  of 
nodes  representing  clusters  of  the  system's 
components:  applications  in  acquaintance  networks  - 
Call  networks,  Protein  interaction  networks  -Internet 

graphs 

Phase  Transitions  in  Random  Graphs 
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First-Order  Phase  Transitions  in  Random  Networks 


P.  Krokhmal  (U.  Iowa),  V.  Boginski,  A.  Veremyev  (UFL),  D.  Jeffcoat  (AFRL) 


Phase  transitions  are  phenomena  in 
large  scale  systems  when  system’s 
properties  change  drastically  once  its 
size  reaches  certain  threshold  value 

Existing  literature  describes 
continuous  (second-order)  phase 
transitions  in  random  graphs 

We  report  a  discontinuous  (first-order) 
phase  transition  in  the  property  of 
dense  connectivity  of  random  graphs 


(A)  A  network  with  20  nodes 
and 22 links.  For/ =  0.7, the 
largest  quasi-clique  (at  least 
70%-dense  subgraph)  has 
size  4. 


Densely  connected  components  of 
graphs  are  modeled  via  quasi-cliques, 
or  subgraphs  of  density  at  least  y  <  1 


(B)  Network  from  (A)  where 
10  more  links  have  formed 
(e.g.,  due  to  increase  of 
probability  p).  Highlighted  are 
quasi-cliques  with  y  =  0.7. 
Their  sizes  are  larger,  but  still 
significantly  smaller  than  the 
size  of  the  network 


•  A  popular  G(/?,p)  model  of  random 
graphs  is  used,  where  a  link  between 
two  nodes  exists  with  probability  0  <  p 
<1 
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First-Order  Phase  Transitions  in  Random  Networks 


P.  Krokhmal  (U.  Iowa),  V.  Boginski,  A.  Veremyev  (UFL),  D.  Jeffcoat  (AFRL) 


As  probability  p  of  links  in  random 
G(n,p)  graph  increases,  the  size  M„ 
of  the  largest  quasi-clique  is  log 
small  in  the  graph  size,  as  long  as  p 
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But,  Mn  undergoes  discontinuous 
jump  ( first-order  phase  transition) 
at  p  =  y: 

M 

lim  lim — —  —  0,  w.h.p. 

pU  y  n  >qo  ^ 

lim  lim  ^ n  =  1,  w.h.p. 

pU  y  n  >  qo  22 


P 


A 


- n=50G 


- r=1000 


- n=50D0 


- n=10DOO 

- n =20000 


(A)  Relative  sizes  of  quasi-cliques  in  small-to-average- 
sized  random  graphs  (note  a  continuous  transition 
becoming  more  abrupt  as  graph  size  increases) 


(B)  Relative  sizes  of  quasi-cliques  in  extremely  large 
random  graphs  (note  the  discontinuous  jump) 


This  is  one  of  first  ls,-order  phase 
transitions  in  uniform  random 
graphs  reported  in  the  literature 

AV,  VB,  PK,  DJ  “Dense  percolations  in  large-scale  mean-field  networks  is  provably  explosive”,  PLOS  ONE 
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Polyomino  Tiling 

O.  Prokopyev  (U.  Pittsburgh) 


•  Application: 

*  Time  delay  control  of  phased  array  systems 

*  Jointly  supported  with  Dr.  Nachman  of 
AFOSR/RSE,  motivated  by  work  at  the 
Sensors  Directorate: 

•  Irregular  Polyomino-Shaped  Subarrays  for  Space-Based 
Active  Arrays  (R.J.  Mailloux,  S.G.  Santarelli,  T.M.  Roberts,  D. 
Luu) 
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Optimization  for  Irregular 
Polyomino  Tilings 


■  Motivation:  Mailloux  et  al.  [2006,  2009]  showed  that  a  phased  array  of  “irregularly”  packed 
polyomino  shaped  subarrays  has  peak  quantization  sidelobes  suppressed  compared  to  an 
array  with  rectangular  periodic  subarrays 

■  Sample  simulation:  rectangular  (regular)  subarray  vs.  irregular  subarray  -  peak  sidelobes 
are  significantly  reduced 


■  Goal:  arrange  polyomino-shaped  subarrays  to  maximize  “irregularity” 

■  Approach:  nonlinear  mixed  integer  programming  (NMIP),  where  “irregularity”  is  modeled 
based  on  information  theoretic  entropy 


I 
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■  Example:  rectangular  (regular)  tiling  vs.  irregular  20x20  tiling  using 
pentomino  family  (each  shape  contains  exactly  5  unit  squares) 

■  “Irregularity”  is  measured  using  information  theoretic  entropy:  minimum  vs. 
maximum  entropy  in  the  example  above 

■  Results:  we  developed  exact  and  approximate  solution  methods 

S  exact  methods  -  solve  instances  of  size  up  to  80x100  in  a  reasonable 
time  (it  depends  on  polyomino  family) 

S  heuristic  and  approximate  methods  -  thousands  (limitations  mainly  on 
storage  memory) 
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•  Goals : 

*  Cutting  edge  optimization  research 

*  Of  future  use  to  the  Air  Force 

*  Strategies : 

*  Stress  rigorous  models  and  algorithms  ,  especially  for 
environments  with  rapidly  evolving  and  uncertain  data 

*  Engage  more  recognized  optimization  researchers,  including 
more  young  Pis 

*  Foster  collaborations  - 

*  Optimizers,  engineers  and  scientists 

*  Between  Pis  and  AF  labs 

*  With  other  AFOSR  programs 

*  With  NSF,  ONR,  ARO 
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